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Although  dramatic  increases  in  the  use  of  automation  in  recent  years 
have  occurred  across  society,  research  has  found  that  human 
operators  often  under-use  (disuse)  and  overly  rely  on  (misuse) 
automated  aids  (Parasuraman  &  Riley,  1997).  A  general  framework  of 
automation  use,  which  proposes  that  cognitive,  social,  and 
motivational  processes  may  lead  to  productivity  loss  of  human- 
computer  teams,  is  developed,  described,  and  defended  with 
anecdotal  and  experimental  findings.  More  specifically,  automation 
use  is  predicted  to  be  affected  by  the  difference  between  the  reliability 
of  the  automated  aid  and  the  reliability  of  manual  operation,  task 
difficulty,  the  number  of  tasks,  interest  in  the  task,  fatigue,  cognitive 
overhead,  the  rewards  for  successful  performance,  the  penalties  for 
unsuccessful  performance,  and  several  cognitive  biases.  Suggestions 
for  future  research  are  briefly  discussed. 
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A  FRAMEWORK  OF  AUTOMATION  USE 


In  December  1995,  the  crew  of  an  American  Airlines  Boeing  757,  descending  through  a 
mountain  valley  toward  Cali,  Columbia,  attempted  to  route  the  aircraft  toward  their 
destination  by  entering  into  the  flight  management  system  (FMS)  a  substring  of  the  code 
for  a  Cali  navigational  beacon.  The  computer's  stored  database  of  navigational  beacons 
contained  two  very  similar  codes.  One  code  denoted  the  beacon  near  Cali,  which  was 
several  dozen  miles  ahead  of  the  airplane.  The  other  code  corresponded  to  a  beacon  at  the 
Bogota  airport,  several  dozen  miles  behind  the  airplane.  Presented  by  the  FMS  with  a  list 
of  nearby  beacons  matching  the  entered  substring,  the  crew  initiated  an  overlearned 
behavior  and  selected  the  computer's  first  presented  alternative;  unfortunately,  the  FMS 
had  presented  the  Bogota  beacon  first.  The  flight  management  computer  dutifully  began 
turning  the  aircraft  toward  Bogota.  Shortly  after  this,  the  aircraft  crashed  into  the  side  of 
a  mountain,  killing  all  on  board  (Phillips,  1999). 

Information  technology  is  changing  the  nature  of  work.  Access  to  more  relevant, 
accurate,  and  timely  information  than  was  previously  possible  has  drastically 
increased  the  use  of  automation  across  society,  including  the  military  (Cesar, 
1995).  The  requirement  for  automation  is  highlighted  by  the  increasing 
complexity  of  the  military  mission,  which  is  expanding  to  include  the  full 
spectrum  of  conflict,  from  humanitarian  assistance  and  peace  keeping  to  small- 
scale  contingencies  and  major  wars.  To  enhance  situational  awareness  and  a 
myriad  of  other  tasks,  the  military  decision  maker  is  often  provided  with 
automated  aids.  The  underlying  assumption  in  providing  these  automated  aids 
to  military  personnel  is  that  the  human-computer  "team"  will  be  more 
productive  than  either  the  human  or  the  automated  aid  would  be  working  alone. 
Some  researchers  have  found  support  for  this  underlying  assumption  (Corcoran, 
Dennett,  &  Carpenter,  1972;  Dalai  &  Kasper,  1994;  Parasuraman,  1987;  Thackray 
&  Touchstone,  1989a).  However,  as  the  previous  vignette  illustrates,  human- 
computer  teams  do  not  always  function  optimally. 

One  strategy  used  to  optimize  human-computer  performance  has  been  to  ask 
system  designers  to  create  automated  aids  that  are  increasingly  more  reliable. 
Increasing  the  automated  aid's  reliability  is  assumed  to  lead  to  increased  human- 
computer  "team"  performance.  As  with  human  teams,  however,  increasing  the 
reliability  of  one  team  member's  performance  will  not  necessarily  affect  the 
team's  performance.  Sorkin  and  Woods'  (1985)  signal  detection  analysis  revealed 
that  optimizing  an  automated  aid's  performance  would  not  always  optimize  the 
human-computer  team's  performance  of  a  monitoring  task.  Specifically,  they 
found  that  using  a  response  criterion  that  yielded  the  best  performance  for  the 
automated  aid  (i.e.,  highest  detection  rates  and  fewest  false  alarms)  did  not  yield 
the  best  performance  for  the  human-automated  team.  Although  "synergy"  can 
be  found  in  human-computer  teams  (e.g.,  Dalai  &  Kasper,  1994),  it  is  not  likely  to 
be  gained  by  optimizing  human-alone  or  computer-alone  performances.  The 
interaction  between  the  automated  aid  and  human  operator  must  be  considered. 
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By  understanding  the  processes  that  human  operators  use  when  deciding  to  rely 
on  their  decisions  or  on  an  automated  aid's  decisions,  one  may  be  able  to  better 
design  systems  and  train  military  decision  makers  about  the  appropriate  use  of 
automated  aids.  The  purpose  of  this  report  is  to  present  a  general  framework  we 
believe  will  be  useful  in  guiding  research  to  understand  human  operators'  task 
allocation  decisions. 


1.  Disuse 


A  rational  decision  maker  will  rely  on  an  automated  aid  when  doing  so  will 
maximize  gains  and/or  minimize  losses.  Failure  to  rely  on  an  aid  in  this  situation 
constitutes  disuse  (Parasuraman  &  Riley,  1997).  Disuse  is  defined  as 
"underutilization  of  automation"  (p.  233).  Anecdotal  evidence  supports  disuse; 
Parasuraman  and  Riley  (1997)  described  many  real-world  incidences  in  which 
disastrous  results  occurred  because  people  ignored  automated  warning  signals. 
Further,  laboratory  experiments  have  found  disuse  in  paradigms  in  which  the 
aid's  decisions  are  provided  only  after  the  human  operators  have  indicated  their 
decision  (Dzindolet,  Pierce,  Beck,  &  Dawe,  1999;  Moes,  Knox,  Pierce,  &  Beck, 
1999).  For  example,  Moes  and  others  (1999)  found  that  most  college  students 
(67%)  chose  to  ignore  the  decisions  of  an  automated  aid  even  after  being 
provided  with  feedback  that  their  automated  aid  made  half  as  many  errors  as 
they  did  during  200  prior  trials.  In  addition,  Riley  (1996)  found  that  students 
favored  manual  operation  over  automated  control,  even  when  doing  so  was 
clearly  not  an  optimal  strategy. 


2.  Misuse 


When  ignoring  an  automated  aid  will  maximize  gains  and/or  minimize  losses, 
the  rational  decision  maker  will  ignore  the  aid  and  rely  on  his  or  her  decisions. 
Relying  on  the  automated  aid  in  this  circumstance  would  constitute  misuse 
(Parasuraman  &  Riley,  1997).  Misuse  is  defined  as  "over-reliance  on  automation" 
(p.  233).  Parasuraman,  Molloy  and  Singh  (1993)  and  Singh,  Molloy,  and 
Parasuraman  (1997)  found  misuse  among  operators  performing  monitoring 
functions.  They  labeled  this  behavior  complacency,  "a  psychological  state 
characterized  by  a  low  index  of  suspicion"  (Wiener,  1981,  p.  809).  Misuse  has  also 
been  found  with  automated  decision  aids  (Dzindolet,  Pierce,  Beck,  Dawe,  & 
Anderson,  to  be  published;  Layton,  Smith,  &  McCoy,  1994;  Mosier  &  Skitka, 
1996). 
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3.  Framework  of  Automation  Use 


What  are  the  processes  leading  to  appropriate  use,  misuse,  and  disuse  of  an 
automated  aid?  Mosier  and  Skitka  (1996)  outlined  three  possible  reasons 
(hypotheses)  why  people  inappropriately  use  automation:  (1)  cognitive  miser, 
(2)  authority,  and  (3)  diffusion  of  responsibility.  These  three  hypotheses  parallel 
three  processes:  cognitive,  social,  and  motivational,  which  have  been  implicated 
as  the  causes  of  productivity  loss  found  in  groups  (e.g.,  Mullen,  Johnson,  &  Salas, 

1991) .  Since  many  researchers  have  considered  a  human-computer  "team"  to  be 
a  group  in  which  one  member  happens  not  to  be  human  (e.g..  Bowers,  Oser, 
Salas,  &  Cannon-Bowers,  1996),  we  hypothesize  that  these  same  three  processes 
can  lead  to  sub-optimal  performance  in  human-computer  teams.  The  framework 
of  automation  use  hypothesizes  that  cognitive,  social,  and  motivational  processes 
of  the  human  operator  combine  to  predict  automation  use.  A  discussion  of  each 
process  as  it  relates  to  inappropriate  use  is  presented  in  the  next  three  sections. 

3.1  Cognitive  Processes:  Cognitive  Miser  Hypothesis  -  Automation  Bias 

Mosier  and  Skitka  (1996)  hypothesized  that  people  may  overly  rely  on 
automated  systems  when  making  decisions  because  of  faulty  cognitive 
processing.  In  addition  to  the  large  body  of  literature  examining  errors 
attributable  to  flawed  cognitive  processing  in  individual  decision  making 
(Tversky  &  Kahneman,  1973),  the  social  cognition  literature  is  replete  with 
examples  of  less-than-ideal  cognitive  processing  while  people  work  in  teams  or 
groups.  Errors  and  biases  have  been  identified  in  various  domains  of  social 
psychology  (e.g.,  illusory  correlation,  "in-group"  differentiation,  and  "out¬ 
group"  homogeneity  in  stereotype  formation  [Hamilton  &  Sherman,  1989; 
Mullen  &  Hu,  1989],  the  halo  effect  and  the  negativity  bias  in  impression 
formation  [Skowronski  &  Carlston,  1989],  the  confirmation  bias  and  self¬ 
handicapping  in  attribution  [Arkin  &  Baumgardner,  1985;  Leyens  &  Yzerbyt, 

1992] ).  People  often  adopt  effort-saving  strategies  called  "heuristics"  instead  of 
logically  processing  relevant  pieces  of  information.  Mosier  and  Skitka  (1996, 
p.  205)  coined  the  term  "automation  bias"  to  refer  to  "the  tendency  to  use 
automated  cues  as  a  heuristic  replacement  for  vigilant  information  seeking  and 
processing." 

The  fact  that  the  automated  system  provides  a  decision  may  lead  the  decision 
maker  to  rely  on  this  information  in  a  heuristic  manner  (see  Figure  1).  Rather 
than  going  through  the  cognitive  effort  of  gathering  and  processing  information, 
the  decision  maker  uses  information  supplied  by  the  automated  systems  (Mosier 
&  Skitka,  1996).  Conceivably,  this  may  occur  in  various  degrees.  In  its  most 
extreme  form,  the  decision  reached  by  the  automated  aid  is  immediately 
adopted.  In  a  less  extreme  form,  the  decision  reached  by  the  aid  may  be  given  an 
inappropriately  large  role  in  the  human's  decision-making  process.  For  example. 
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Layton,  Smith,  and  McCoy  (1994)  found  that  many  pilots  provided  with  an 
automated  aid's  poor  en-route  flight  plan  did  not  explore  other  solutions  (e.g., 
they  did  not  generate  actual  flight  plans  on  screen)  as  much  as  pilots  who  were 
not  provided  with  the  automated  aid's  decision. 


Cognitive  Processes 

Provision  of  Aid’s 
Evaluation  Before 
Decision  Making 
(automation  bias) 


Automation  Use 


Figure  1.  Automation  Bias. 


The  automation  bias  will  lead  participants  to  rely  on  the  automated  aid.  Often, 
this  strategy  will  be  appropriate.  However,  during  certain  conditions,  this 
reliance  may  be  inappropriate,  leading  to  misuse. 

Although  the  automation  bias  plays  a  role  in  predicting  automation  use,  it 
cannot  account  for  the  full  range  of  potential  findings.  Specifically,  automation 
bias  would  not  be  able  to  account  for  the  anecdotal  evidence  for  disuse  when  an 
automated  aid's  evaluation  is  provided  to  humans  before  they  report  a  final 
decision.  The  information  received  from  the  automated  aid  is  predicted  to 
influence  the  human's  response.  The  human  operator  is  not  predicted  to  ignore 
information  provided  by  the  automated  aid. 

Indirect  evidence  to  support  the  existence  of  automation  bias  can  be  found  by 
examining  task  allocation  decisions  across  two  studies  with  similar  procedures 
(Dzindolet  et  al.,  to  be  published;  Dzindolet  et  al.,  1999). 

Participants  viewed  about  200  slides  displaying  pictures  of  Fort  Sill  terrain  on  a 
computer  screen  (see  Figure  2  for  a  sample  slide).  About  half  of  the  slides 
contained  one  soldier  ("target")  in  various  levels  of  camouflage;  the  remaining 
slides  were  of  terrain  only. 

Sometimes,  the  target  was  easy  to  detect  (as  in  Figure  2);  other  times,  it  was  more 
difficult  to  find.  Each  slide  was  presented  on  the  computer  screen  for  about  3 /4 
of  a  second.  In  the  experiment  by  Dzindolet  and  others  (to  be  published), 
participants  were  provided  with  the  absent  or  present  evaluation  of  an 
automated  aid  known  to  be  accurate  60%,  75%,  or  90%  of  the  time  (based  on  the 
condition  to  !  which  the  participant  was  randomly  assigned).  Specifically, 
participants  wjere  told  that  a  computer  routine  had  been  written  to  assist  them  in 
performing  their  task.  They  were  told  that  the  routine  performed  a  rapid  scan  of 
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the  photograph  and  looked  for  contrasts  that  suggested  the  presence  of  a  human 
being.  If  the  contrast  detector  routine  determined  the  soldier  was  probably 
present,  the  word  "present"  and  a  red  circle  appeared.  If  the  contrast  detector 
routine  determined  the  soldier  was  probably  absent,  the  word  "absent"  and  a 
green  circle  appeared. 


Figure  2.  Sample  Slide  Showing  Soldier. 


The  next  screen  asked  the  students  to  indicate  whether  they  believed  the  soldier 
was  in  the  slide.  They  were  told  that  they  had  as  much  time  as  they  needed  to 
make  their  decision.  Participants  were  informed  that  there  were  two  types  of 
errors  that  could  be  made.  One  error  was  made  when  they  indicated  that  the 
soldier  was  present  when  he  was  not.  The  other  error  was  made  when  they 
indicated  that  the  soldier  was  not  present  when  he  actually  was.  Participants 
were  told  that  both  errors  were  equally  serious  and  that  they  should  attempt  to 
avoid  them. 

Finally,  the  participants  were  asked  to  indicate  the  extent  to  which  they  were 
certain  their  decision  was  correct.  A  five-point  Likert  scale  ranging  from  "highly 
confident"  to  "not  at  all  confident"  was  provided. 

Analyzed  with  principles  from  signal  detection  theory,  the  error  rate  was 
determined  to  be  the  complement  of  the  area  under  the  receiver  operator  curve. 
To  examine  misuse  and  disuse,  the  probability  of  error  was  determined  for  two 
subsets  of  the  data.  Misuse,  or  over-reliance  on  the  contrast  detector,  was 
operationally  defined  as  the  j>(error  I  aid  error).  Disuse,  or  underutilization  of 
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the  contrast  detector,  was  operationally  defined  as  the  j>(error  I  aid  correct). 
Analyses  indicated  that  regardless  of  the  reliability  of  the  automated  aid,  each 
participant  was  more  likely  to  err  by  relying  on  his  decision  aid  than  by  ignoring 
it,  pferror  I  aid  error)  =  .27;  g(error  I  aid  correct)  =  .13. 

Therefore,  when  provided  with  the  aid's  decisions  first,  thereby  allowing  the 
automation  bias  to  occur,  participants  were  more  likely  to  misuse  than  disuse 
their  automated  aids. 

However,  in  the  studies  by  Dzindolet  and  others  (1999),  the  automation  bias  was 
eliminated.  Participants  were  not  provided  with  the  decisions  reached  by  the 
contrast  detector  until  after  they  had  indicated  their  decision  and  their  level  of 
confidence  in  their  decision.  Without  the  automation  bias,  would  participants 
still  be  more  likely  to  misuse  than  disuse  their  automated  aids? 

Participants  in  these  studies  viewed  the  slide  of  Fort  Sill  terrain  for  about  3/4  of  a 
second,  indicated  their  decision,  rated  their  confidence,  and  only  then  were 
provided  with  the  contrast  detector's  decision.  When  they  completed  all  200 
slides,  some  participants  were  told  the  number  of  errors  they  and  their  aid  made. 
Finally,  students  were  told  they  could  earn  $0.50  in  coupons  to  be  used  at  the 
cafeteria  in  the  student  union  for  every  correct  decision  made  in  ten  randomly 
chosen  trials.  Participants  had  to  choose  whether  the  performance  would  be  based 
on  their  decisions  or  on  the  decisions  of  their  aid.  After  making  their  choice, 
students  were  asked  to  justify  their  choice  in  writing. 

Rather  than  misusing  the  contrast  detector,  participants  in  these  studies  disused 
the  automated  aid.  Even  among  participants  provided  with  feedback  that  their 
aid's  performance  was  far  superior  to  their  own,  the  majority  (81%)  chose  to  rely 
on  their  own  decisions  rather  than  on  the  decisions  of  the  automated  aid! 

Therefore,  when  the  automation  bias  could  play  a  role  in  the  decision  to  rely  on 
automation,  misuse  occurred  more  than  disuse  (Dzindolet  et  al.,  to  be  published). 
However,  when  the  automation  bias  was  eliminated  by  the  provision  of  the 
automated  aid's  decisions  only  after  participants  recorded  their  decision,  disuse 
(not  misuse)  was  found  in  a  subsequent  task  allocation  decision  (Dzindolet  et  al., 
1999). 

Future  research  should  be  directed  at  understanding  the  automation  bias  and 
identifying  variables  that  affect  it.  The  results  from  Dzindolet  and  others  (to  be 
published)  imply  that  the  reliability  of  the  automated  system  does  not  affect  the 
automation  bias.  However,  in  the  study,  the  reliability  information  was  provided 
to  participants  early  in  their  interaction  with  the  system.  If  the  reliability 
information  had  been  provided  continually  throughout  the  task  (e.g.,  feedback 
after  every  trial),  perhaps  automation  reliability  would  have  affected  the 
automation  bias.  Future  research  should  examine  this  and  other  variables  that 
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affect  the  automation  bias  in  an  effort  to  reduce  the  bias  and  increase  the 
likelihood  that  human  operators  will  appropriately  rely  on  automated  systems. 

Although  we  hypothesize  that  automation  bias  plays  a  role  in  automation  use 
and  misuse,  we  do  not  believe  it  is  the  only  important  variable.  At  the  very  least, 
processes  predicting  the  bias  toward  disuse,  which  are  described  in  the  studies 
by  Dzindolet  and  others  (1999),  are  needed.  Analyses  of  the  justifications  of  the 
task  allocation  decisions  provided  by  participants  in  one  of  the  experiments  by 
Dzindolet  and  others  (1999)  indicated  that  other  variables  might  play  a  role.  For 
example,  nearly  1/4  (23%)  of  the  participants  justified  their  disuse  by  stating  that 
they  did  not  trust  the  automated  aid  as  much  as  they  trusted  themselves. 

3.2  Social  Processes:  Authority  Hypothesis  -  Trust  in  Automation 

A  second  explanation  of  the  inappropriate  use  of  automated  aids  has  to  do  with 
the  role  of  the  computer  as  the  expert.  According  to  Mosier  and  Skitka's  (1996) 
authority  hypothesis,  people  rely  on  the  automated  system's  decision  because 
they  believe  it  to  be  more  reliable  and  thus  place  greater  trust  in  it.  Many  other 
researchers  have  indicated  that  trust  is  one  important  variable  in  predicting 
automation  use  (Cohen,  Parasuraman,  &  Freeman,  1998;  Lee  &  Moray,  1992, 
1994;  Moray,  Inagaki,  &  Itoh,  2000;  Muir,  1987,  1994;  Singh,  Molloy,  & 
Parasuraman,  1993;  Tan  &  Lewandowsky,  1996).  Overly  trusting  an  automated 
aid  will  lead  human  operators  to  misuse;  lack  of  trust  in  a  superior  aid  will  lead 
to  disuse. 

Parasuraman  and  Riley  (1997)  described  many  real-world  incidences  in  which 
disastrous  results  occurred  because  people  ignored  automated  warning  signals 
they  saw  as  untrustworthy.  Ignoring  automated  alarm  systems  that  have 
previously  signaled  erroneously  has  been  dubbed  the  "cry  wolf"  effect  (Bliss, 
1997;  Bliss,  Dunn,  &  Fuller,  1995).  Publicizing  the  alarm's  trustworthiness  was 
one  strategy  that  proved  effective  in  reducing  the  cry  wolf  phenomenon  (Bliss  et 
al.,  1995). 

Muir  (1987, 1994),  one  of  the  first  researchers  to  focus  on  automation  trust,  relied 
on  the  literature  of  human  trust  to  understand  the  human  operator's  trust  of 
automation.  She  hypothesized  that  automation  will  be  seen  as  more  trustworthy 
if  it  is  predictable,  dependable,  and  inspires  faith  that  the  automated  system  will 
behave  as  expected  in  unknown  situations.  Trust  is  gained  in  the  areas  of 
persistence,  technical  competence,  and  fiduciary  responsibility. 

To  test  some  of  Muir's  hypotheses,  Lee  and  Moray  (1992,  1994)  examined  the 
effect  of  trust  in  automation  on  task  allocation  decisions.  Participants  controlled  a 
simulated  orange  juice  pasteurization  plant  for  2  hours  each  day  for  three  days. 
The  simulation  included  three  subsystems,  each  of  which  could  be  operated 
manually  or  automatically.  Participants  could  allocate  tasks  any  way  they 
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wished  and  could  change  their  allocations  easily.  As  part  of  the  experiment, 
whether  controlled  automatically  or  manually,  one  of  the  subsystems  failed 
periodically.  Lee  and  Moray  (1992,  1994)  were  especially  interested  in  task 
allocation  changes  after  these  failure  events,  since  Muir  predicted  that  after 
failure  events,  trust  would  decline  rapidly  and  slowly  increase  as  the  system 
performed  without  making  errors.  At  the  end  of  each  session,  participants 
completed  a  questionnaire  about  their  trust  in  the  automation  and  self- 
confidence  in  performing  the  tasks  manually. 

Results  indicated  strong  individual  differences  in  automation  use.  Some 
participants  were  prone  to  use  manual  control;  others  were  prone  to  use 
automation.  Singh  and  others  (1993)  created  a  scale  to  determine  individual 
differences  in  the  propensity  to  misuse  automated  aids,  which  may  have  been 
able  to  predict  which  individuals  would  tend  to  rely  on  automation  and  which 
would  tend  to  rely  on  manual  operation. 

Inconsistent  with  Muir's  hypotheses,  the  Lee  and  Moray  (1992, 1994)  participants 
rarely  changed  their  allocation  task  decisions.  Once  the  human  operator  assigned 
a  subsystem  to  automated  or  manual  control,  he  or  she  was  unlikely  to  change 
the  decision  during  that  session — even  after  failure  events. 

To  predict  trust  in  automation  dynamically,  Cohen,  Parasuraman,  and  Freeman 
(1998)  extended  Muir's  (and  others)  work  and  created  an  especially  promising 
model.  Argument-based  Probabilistic  Trust  (APT).  This  model  defines  trust  as 
the  perceived  probability  of  the  system's  reliability,  given  certain  situations. 
Instead  of  trust  being  correlated  with  the  predicted  reliability  of  the  system  (e.g., 
Muir's  1994  theory),  trust  is  the  predicted  reliability  of  the  system. 

Whether  predicted  reliability  is  related  to  trust  (e.g.,  Muir,  1994)  or  is  trust  (e.g., 
APT  model),  we  do  not  believe  that  predicted  reliability  will  directly  predict 
automation  use.  The  predicted  reliability  of  an  automated  aid  is  meaningful  only 
in  the  context  of  some  standard  of  performance.  What  standard  did  Lee  and 
Moray's  (1992,  1994)  subjects  use?  Automation  use  was  found  to  be  related  to 
subjects'  estimates  of  the  ability  of  the  automated  aid  relative  to  estimates  of  their 
own  ability.  Thus,  we  hypothesize  that  automation  use  is  determined  from  the 
outcome  of  a  comparison  process  between  the  perceived  reliability  of  the 
automated  aid  (trust  in  aid)  and  the  perceived  reliability  of  manual  control  (trust 
in  self).  We  call  the  outcome  of  the  decision  process  the  perceived  utility  of  the 
automated  aid  (see  Figure  3).  If  one  perceives  the  ability  of  the  aid  to  be  greater 
than  one's  own,  perceived  utility  of  the  aid  will  be  high.  If  one  perceives  the 
ability  of  the  aid  to  be  inferior  to  one's  own,  perceived  utility  of  the  aid  will  be 
low.  People  may  misuse  an  automated  aid  because  the  perceived  utility  of  the 
aid  is  overestimated;  they  may  disuse  an  aid  when  the  perceived  utility  of  the  aid 
is  underestimated. 
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Figure  3.  Relative  Trust. 


Since  accurately  perceiving  the  utility  of  the  aid  will  lead  to  appropriate 
automation  use,  it  is  very  important  that  we  gain  an  understanding  of  how  this 
perception  is  formed.  The  perceived  utility  of  the  aid  will  be  most  accurate  when 
the  actual  ability  of  the  aid  and  actual  ability  of  the  manual  operator  are 
compared.  However,  the  "actual"  ability  is  never  known.  Perceived  ability  is 
determined  through  a  function  of  actual  ability  and  error.  The  larger  the  error, 
the  more  likely  misuse  and  disuse  is  to  occur.  We  suspect  that  at  least  two  types 
of  errors  occur. 

One  type  of  error  occurs  when  human  operators  estimate  their  own  performance. 
Human  operators  tend  to  overestimate  their  own  ability.  Social  psychological 
literature  is  fraught  with  examples  of  self-serving  biases.  Humans  exaggerate 
their  contribution  to  a  group  product  (appropriation  of  ideas,  Wicklund,  1989), 
overestimate  the  number  of  tasks  they  can  complete  in  a  given  period  of  time 
(planning  fallacy,  Buehler,  Griffin,  &  Ross,  1994),  are  overconfident  in 
negotiations  (Neale  &  Bazerman,  1985),  and  inflate  their  role  in  positive 
outcomes  (Whitley  &  Frieze,  1985).  Thus,  we  hypothesize  that  human  operators 
will  be  likely  to  overestimate  their  manual  ability. 

Providing  accurate  feedback  will  assuage  the  self-serving  biases.  Using  the 
paradigm  of  Dzindolet  and  others  (1999)  described  earlier,  Dzindolet,  Beck,  and 
Pierce  (2000)  found  that  providing  feedback  of  the  contrast  detector's  superior 
performance  after  each  trial  and/or  at  the  end  of  the  200  trials  reduced  disuse. 

The  other  type  of  error  occurs  when  human  operators  estimate  the  performance 
of  their  automated  aid.  Before  working  with  the  aid,  the  human  must  rely  on 
stereotypes  formed  about  the  performance  of  automated  aids.  Although 
individual  differences  exist,  a  bias  toward  automation  leads  many  people  to 
predict  nearly  perfect  performances  from  automated  aids.  Dzindolet  and  others 
(1999)  told  half  the  participants  that  they  would  be  provided  with  the  decisions 
reached  by  the  contrast  detector  before  they  made  their  soldier  absent  or  present 
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decision  for  each  of  200  slides.  Other  participants  were  told  that  they  would  be 
provided  with  the  decisions  reached  by  the  previous  participant  before  making 
their  soldier  absent  or  present  decision  for  each  of  the  200  trials.  The  instructions 
informed  the  participants  that  their  aid  (human  or  automated)  was  not  perfect. 
When  asked  to  estimate  the  number  of  errors  that  the  automated  aid  would 
make  in  the  imminent  200  trials,  participants  predicted  that  the  aid  would  make 
an  average  of  only  14.19  errors  (i.e.,  correct  nearly  93%  of  the  time).  When  asked 
to  estimate  the  number  of  errors  that  the  human  aid  would  make,  participants 
predicted  an  average  of  46.17  errors  (i.e.,  correct  only  77%  of  the  time-only  27% 
better  than  chance!).  Although  previous  researchers  have  reported  that  people 
tend  to  have  negative  initial  expectations  of  automation  (e.g.,  Halpin,  Johnson,  & 
Thomberry,  1973),  our  studies  reveal  a  strong  bias  toward  automation.1 

In  summary,  the  perceived  reliability  of  the  automated  aid  is  determined  by  the 
actual  reliability  of  the  automated  aid  and  by  the  bias  toward  automation.  The 
perceived  reliability  of  manual  operation  is  determined  by  the  actual  reliability 
of  the  human  operator  and  by  self-serving  biases. 

We  hypothesize  that  what  is  important  in  determining  automation  use,  however, 
is  not  the  perceived  reliability  of  the  aid  or  the  perceived  reliability  of  manual 
control  but  the  result  of  a  comparison  process  between  the  two:  perceived 
utility.  Increasing  the  reliability  of  the  aid  will  not  increase  automation  use 
unless  the  aid's  perceived  reliability  surpasses  that  of  manual  operations. 

This  can  explain  the  inconsistent  findings  concerning  automation  reliability  and 
human-computer  performance.  For  example,  Riley  (1994)  and  Moray,  Inagaki, 
and  Itoh  (2000)  found  that  people  were  more  likely  to  overly  rely  on  a  more 
reliable  automated  system  than  on  a  less  reliable  one.  Yet,  Dzindolet  and  others 
(to  be  published)  did  not  find  that  participants  rely  on  more  reliable  decision  aids 
than  on  less  reliable  ones. 

Similarly,  Parasuraman  and  others  (1993)  and  Singh  and  others  (1997)  did  not 
find  reliability  to  affect  automation  use  with  a  monitoring  task.  Participants  were 
required  to  simultaneously  perform  three  flight  simulation  tasks.  Two  of  the 
tasks  required  manual  operation,  but  a  third  task  (monitoring)  was  automated. 
The  automation  varied  in  its  reliability  and  in  its  variability.  Some  participants 
worked  with  an  automated  system  that  was  consistently  correct  87.5%  of  the 
time;  others  worked  with  a  system  that  was  consistently  correct  56.25%  of  the 


'Confusion  could  arise  from  the  similarity  of  the  term,  bias  toward  automation,  to  a  term  discussed 
earlier,  automation  bias.  Bias  toward  automation  refers  to  the  finding  that  participants  initially 
expect  automated  aids  to  perform  better  than  human  operators  (Dzindolet  et  al.,  1999).  The 
automation  bias  is  a  term  coined  by  Mosier  and  Skitka  (1996)  to  refer  to  "the  tendency  to  use 
automated  cues  as  a  heuristic  replacement  for  vigilant  information  seeking  and  processing"  (p. 
205).  In  fact,  the  cues  (i.e.,  the  decisions)  would  not  have  to  come  from  an  automated  aid.  We 
would  expect  the  decisions  of  both  human  and  automated  aids  to  be  relied  on  in  a  heuristic 
manner. 
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time.  Others  worked  with  machines  that  varied  in  reliability.  Some  students 
started  with  automated  systems  that  were  accurate  87.5%  of  the  time  for  the  first 
10  minutes  of  the  session  of  the  experiment  but  then  alternated  in  reliability  to 
56.25%  and  87.5%  during  10-minute  intervals  for  the  rest  of  the  experimental 
session.  Other  participants  started  working  with  a  less  reliable  automated  system 
that  alternated  to  a  more  reliable  every  10  minutes  during  the  experimental 
session. 

Over-reliance  on  automation  was  found  by  participants  in  both  of  the  constant 
conditions  (Parasuraman  et  al.,  1993),  and  the  saliency  of  the  automation  failure 
signals  did  not  affect  the  results  (Singh  et  al.,  1997).  Thus,  when  the  reliability  of 
the  automation  varied,  participants  appropriately  relied  on  the  automated 
monitoring  system.  However,  when  the  system  was  consistent  in  its  reliability 
(either  high  or  low),  participants  tended  to  overly  rely  on  the  system's 
monitoring  capabilities.  Singh  and  others  (1997)  hypothesize  that  complacency 
was  attributable  to  the  participants  being  too  trusting  of  the  automation. 
"Participants  may  have  begun  the  automated  sessions  with  an  equal  amount  of 
trust  in  the  automation.  However,  as  the  reliability  of  the  automation  fluctuated 
for  tile  variable  group,  their  trust  may  have  declined.  Therefore,  the  variability 
group  may  have  been  more  skeptical  of  the  automation  and  thus  been  more 
vigilant  for  automation  failures"  (p.  28). 

Why  did  reliability  not  affect  automation  use  in  the  studies  of  Parasuraman  and 
others  (1993)  and  Singh  and  others  (1997)?  This  finding  is  unexpected  according 
to  the  APT  model  and  Muir's  model  of  trust.  Our  model  predicts  that 
automation's  reliability  will  affect  automation  use  only  if  it  leads  the  human 
operator  to  perceive  the  aid's  reliability  as  greater  than  his  or  her  own. 

In  summary,  we  hypothesize  that  people  may  misuse  an  automated  aid  when  the 
perceived  utility  of  the  aid  is  overestimated  and  disuse  an  aid  when  the 
perceived  utility  of  the  aid  is  underestimated.  The  perceived  utility  of  the  system 
results  from  a  comparison  between  the  automated  system's  perceived  ability  and 
one's  own  perceived  ability.  Perceived  ability  is  hypothesized  to  be  affected  by 
actual  ability  and  various  biases  (self-serving  and  bias  toward  automation). 

Reducing  the  biases  should  decrease  inappropriate  automation  use.  Dzindolet 
and  others  (to  be  published)  found  that  disuse  could  be  reduced  by  providing 
participants  multiple  forms  of  feedback  of  the  aid's  superior  performance. 

In  addition  to  trust  (through  perceived  utility)  affecting  automation  use,  we 
conjecture  that  two  other  social  processes  may  affect  automation  use:  feelings  of 
control  and  moral  obligation  to  rely  on  self.  Figure  4  illustrates  the  social 
processes. 


11 


Figure  4.  Social  Processes. 


Analyses  of  the  justifications  of  the  task  allocation  decisions  provided  by 
participants  in  one  of  the  experiments  by  Dzindolet  and  others  (1999)  revealed 
that  71%  of  the  students  (who  were  provided  cumulative  feedback  that  indicated 
that  the  aid  made  about  an  equal  number  of  errors  as  the  participant)  justified 
self-reliance  with  statements  indicating  they  would  not  earn  more  rewards  if 
they  relied  on  the  aid.  Since  the  task  allocation  decision  would  not  affect  the  size 
of  their  reward,  why  did  participants  choose  self-reliance?  We  hypothesize  that 
self-reliance  provides  participants  with  an  illusion  of  control.  Langer  (1983)  has 
found  that  people  often  behave  illogically  in  order  to  have  an  illusion  of  control. 

In  addition,  many  participants  (although  more  of  whom  worked  with  human 
aids  [n=24,  43.64%]  than  automated  aids  [n=9,  16.67%],  X2  =  8.58,  p  <  .01) 
justified  self-  reliance  with  statements  concerning  a  moral  obligation  to  rely  on 
self.  One  student  wrote,  "I  would  rather  the  amount  of  coupons  I  receive  be 
based  on  my  performance;  it  seems  more  'fair'  to  me."  Another  wrote,  "I  feel 
anything  earned  should  be  based  on  how  well  I  did  or  didn't  do." 

Much  research  remains  to  be  performed  to  explore  the  social  processes  and  their 
effect  on  automation  use.  Only  with  a  more  clear  understanding  of  these 
processes  will  we  be  able  to  suggest  ways  that  misuse  and  disuse  can  be  reduced. 

The  variables  that  affect  perceived  utility  are  of  special  interest  to  us  because 
perceived  utility  is  not  only  predicted  to  affect  trust  but  also  to  affect  the  last  of 
the  processes,  the  motivational  process — that  is,  the  variables  that  affect 
automation  use  by  increasing  or  decreasing  the  amount  of  effort  expended  on  the 
task.  i 
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3.3  Motivational  Processes:  Diffusion  of  Responsibility 

A  third  explanation  of  the  over-reliance  on  automation  discussed  by  Mosier  and 
Skitka  (1996)  involves  the  idea  that  when  people  work  in  a  group,  the 
responsibility  for  the  group's  product  is  diffused  among  the  group  members. 
Several  researchers  have  thought  of  the  human-computer  system  as  a  dyad  or 
team  in  which  one  member  is  not  human  (Bowers  et  al.,  1996;  Scerbo,  1996; 
Woods,  1996).  Thus,  the  human  may  feel  less  responsible  for  the  outcome  when 
working  with  an  automated  system  than  when  working  without  one.  The  person 
may  not  be  as  motivated  to  extend  as  much  effort  when  paired  with  an 
automated  system  as  when  working  alone.  In  the  social  psychological  literature, 
this  phenomenon  has  been  dubbed  "social  loafing"  (Latane,  Williams,  &  Harkins, 
1979)  or  "free  riding"  (Kerr  &  Bruun,  1983). 

One  theory  that  has  been  successful  in  accounting  for  much  of  the  findings  in  the 
social  loafing  literature  is  Shepperd's  Expectancy  Value  Theory  (1993,  1998). 
According  to  this  theory,  motivation  is  predicted  from  a  function  of  three  factors: 
expectancy,  instrumentality,  and  outcome  value. 

3.3.1  Expectancy 

The  first  factor,  expectancy,  is  the  extent  to  which  members  feel  that  their  efforts 
are  necessary  for  the  group  to  succeed  (see  Figure  5).  When  members  feel  their 
contributions  are  dispensable  or  when  one's  individual  contribution  is 
unidentifiable  or  not  evaluated,  one  is  likely  to  free  ride  or  work  less  (Kerr  & 
Bruun,  1983;  Williams  &  Karau,  1991). 


Task  Difficulty 


Perceived 

Utility 


i 


Dispensability 


Figures.  Expectancy. 


With  a  human-computer  system,  individual  contributions  tend  to  be  identifiable 
and  evaluated;  thus,  these  variables  are  not  thought  to  affect  motivational 
processes.  However,  when  the  perceived  utility  of  a  system  is  high,  one  is  likely 
to  feel  that  his  or  her  efforts  are  more  dispensable  than  when  he  or  she  is  when 
working  with  a  system  low  in  perceived  utility.  Thus,  we  would  expect  human 
operators  to  be  likely  to  misuse  an  automated  system  deemed  more  reliable  than 
themselves  in  the  same  way  people  free  ride  on  group  members  deemed  more 
reliable  than  themselves. 
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Task  difficulty,  which  is  predicted  to  affect  perceived  utility  (see  Figure  5),  has 
also  been  found  to  directly  affect  dispensability.  In  fact,  one  of  the  methods 
researchers  have  used  to  make  group  members  feel  that  their  efforts  are 
indispensable  has  been  to  imply  that  the  difficulty  of  the  task  makes  the 
demands  on  each  group  member  particularly  high  (Shepperd,  1993).  For 
example,  Harkins  and  Petty  (1982)  asked  participants  to  generate  as  many  uses 
as  they  could  for  either  a  knife  (easy  task)  or  for  a  detached  door  knob  (difficult 
task)  either  alone  or  in  nine-member  groups.  Although  they  found  social  loafing 
with  the  easy  task  (group  members  did  not  generate  as  many  ideas  as  those 
working  alone),  they  did  not  find  social  loafing  with  the  difficult  task. 

In  summary,  the  more  dispensable  the  human  operator  is  made  to  feel,  the  lower 
expectancy  will  be;  effort  will  probably  be  low  and  the  likelihood  that  the 
automated  aid  will  be  relied  upon  will  be  high.  In  some  instances,  this  will  lead 
to  automation  misuse. 

3.3.2  Instrumentality 

Instrumentality,  the  extent  to  which  members  feel  that  the  group's  successful 
performance  will  lead  to  a  positive  overall  outcome,  is  also  predicted  to  affect 
effort.  Members  who  feel  the  outcome  is  not  contingent  on  the  group's 
performance  are  less  likely  to  work  hard.  Thus,  inappropriate  use  should  be  high 
among  members  who  feel  their  group's  performance  is  irrelevant.  In  one  study, 
Shepperd  (1998)  varied  instrumentality.  Half  the  participants  were  told  that  the 
seven  groups  (of  ten  groups)  with  the  highest  number  of  ideas  generated  would 
earn  a  reward.  Other  participants  were  told  that  the  members  of  the  four  groups 
(of  40  groups)  with  the  highest  number  of  ideas  generated  would  have  their 
names  entered  into  a  lottery.  One  name  would  be  drawn  and  that  person  would 
earn  a  reward.  Thus,  in  the  former  condition,  members  had  7  in  10  chances  of 
attaining  the  reward;  in  the  latter  condition,  there  was  only  1  in  200  chances  of 
attaining  the  reward.  In  addition  to  the  optimistic  bias  (participants  estimated 
their  chance  of  winning  to  be  about  1  in  4),  Shepperd  found  that  performance 
suffered  when  instrumentality  was  lowered. 

On  the  battlefield,  there  will  be  many  soldier-computer  teams.  If  the  human 
determines  that  the  overall  outcome  is  not  contingent  on  his  or  her  human- 
computer  team  (either  because  he  estimates  other  teams  to  be  more  able  to  do  the 
task  or  that  his  human-computer  team  is  dispensable),  then  the  human  will  put 
little  effort  into  the  task. 

3.33  Outcome  Value 

Finally,  the  value  of  the  outcome  is  predicted  to  affect  motivation.  Outcome 
value  is  the  difference  between  the  importance  of  the  outcome  and  the  costs 
associated  with  working  hard.  Increasing  the  costs  or  minimizing  the  importance 
of  file  reward  will  lead  members  to  put  forth  less  effort.  More  effort  will  be 
extended  toward  tasks  that  lead  to  valuable  outcomes  without  requiring  much 
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cost.  Costs  vary  with  the  number  of  other  tasks  one  must  perform,  fatigue, 
intrinsic  interest  in  the  task,  and  cognitive  overhead  (see  Figure  6). 


Figure  6.  Costs. 


3.3.3.1  Number  of  tasks 

As  workload  increases,  the  cost  of  performing  a  specific  task  increases,  thereby 
increasing  automation  use  (and  the  potential  for  misuse).  In  Study  2  of 
Parasuraman  and  others  (1993),  misuse  did  not  occur  among  participants  who 
were  required  only  to  perform  the  automated  task.  Over-reliance  on  automation 
was  found  only  among  participants  who  had  to  manually  perform  two 
additional  tasks.  Similarly,  Thackray  and  Touchstone  (1989b)  did  not  find 
differences  in  detection  of  failures  between  aided  and  unaided  participants 
performing  a  single  task.  Thus,  consistent  with  the  framework,  increasing  the 
number  of  tasks  increased  the  likelihood  for  misuse. 

3.3.3.2  Fatigue 

The  more  fatigued  the  human  operator  is,  the  higher  the  cost  of  performing  an 
additional  task.  Therefore,  we  would  expect  automation  use  to  be  greater  with 
more  fatigued  than  less  fatigued  human  operators.  In  some  instances,  this  will 
lead  to  appropriate  automation  use,  but  in  other  instances,  this  will  lead  to 
misuse  of  the  automated  aid. 

3.3.3.3  Interest  in  the  task 

The  costs  of  performing  a  task  that  is  intrinsically  interesting  are  less  than  the 
costs  of  performing  a  less  interesting  task.  For  example,  the  cost  of  performing 
boring,  redundant  tasks  is  higher  than  the  cost  of  performing  interesting  tasks.  If 
the  task  is  boring  enough,  the  costs  might  outweigh  the  importance;  outcome 
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value  will  be  decreased,  effort  will  be  lower,  and  automation  use  and  the 
potential  for  misuse  will  rise. 

33.3.4  Cognitive  overhead 

If  the  cost  to  change  from  manual  operation  to  automation  use  is  great,  one  is  less 
likely  to  put  forth  the  effort  to  make  the  change.  Parasuraman  and  Riley  (1997) 
indicate  that  high  cognitive  overhead  can  induce  complacency  or  misuse  if 
participants  initially  allocate  the  task  to  the  automated  system.  Similarly,  high 
cognitive  overhead  can  induce  disuse  if  participants  initially  allocate  the  task  to 
manual  operation. 

In  summary,  if  the  costs  of  manual  operation  exceed  importance  because  of  a 
great  number  of  tasks,  fatigue  of  the  human  operator,  boredom  of  the  task, 
and/or  cognitive  overhead,  then  outcome  value  will  be  low.  The  amount  of 
effort  expended  on  the  task  is  predicted  to  be  low.  Thus,  the  likelihood  for 
automation  use  and  the  potential  for  misuse  will  be  great. 

Costs  will  only  affect  automation  use  if  they  are  deemed  greater  than  the 
importance  of  the  outcome.  If  the  importance  of  the  outcome  is  deemed  greater 
than  the  cost,  outcome  value  will  be  high,  thereby  increasing  effort.  The 
likelihood  for  automation  use  is  low  in  this  situation,  making  the  possibility  for 
disuse  potentially  large. 

Importance  is  predicted  to  be  affected  by  the  rewards  of  successful  task 
completion  and  the  penalties  of  task  failure  (see  Figure  7). 


Figure  7.  Importance  of  Outcome. 


33.3.5  Rewards 

When  successful  completion  of  the  task  leads  to  highly  valued  resources  (e.g., 
money,  prestige),  and  if  the  rewards  are  not  outweighed  by  the  costs  of 
succeeding,  then  outcome  value  will  be  high  and  participants  are  predicted  to 
work  hard.  Human  operators  will  not  rely  on  the  automated  aid;  rather,  they  will 
put  forth  the  Effort  to  complete  the  task.  Often,  this  strategy  will  be  appropriate; 
sometimes,  disuse  will  occur,  however. 
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3.3.3.6  Penalties 


Similarly,  when  grave  penalties  are  a  consequence  of  unsuccessful  completion  of 
the  task,  specifically  when  the  penalties  of  failure  outweigh  the  costs  of 
succeeding,  then  outcome  value  will  be  high  and  participants  are  predicted  to 
work  hard.  In  some  situations,  this  will  lead  to  an  optimal  task  allocation 
decision;  sometimes,  it  will  lead  to  disuse  of  automated  aids. 

On  the  battlefield,  sub-optimal  human-computer  performance  can  be  lethal; 
outcome  value  is  extremely  high.  In  combat,  disuse  may  be  more  of  a  problem 
than  misuse.  Among  such  highly  motivated  people,  misuse  may  not  exist  at  all! 
This  is  consistent  with  findings  from  some  interviews  with  Gulf  War  soldiers 
who  indicated  that  they  turned  off  their  automated  systems. 

For  this  reason,  it  is  imperative  that  some  of  the  research  testing  of  the  model  be 
performed  in  more  combat-like  environments.  At  the  very  least,  researchers 
should  examine  automation  reliance  while  varying  the  consequences  for 
successful  task  performance.  Although  it  is  intuitively  appealing,  albeit 
somewhat  obvious,  to  predict  that  instituting  positive  consequences  for 
successful  performance  would  lead  to  improved  automation  reliance,  this  may 
not  necessarily  occur.  In  Studies  2  and  3  of  Dzindolet  and  others  (1999)  and 
experiments  by  Moes  and  others  (1999),  participants  earned  coupons  or  extra 
credit  for  correct  decisions.  Yet,  in  all  three  studies,  disuse  prevailed.  Of  course, 
in  these  studies,  the  consequences  were  not  varied.  We  predict  that  increasing 
the  rewards  and/or  the  penalties  for  correct  decisions  would  have  decreased 
disuse.  Future  research  needs  to  determine  the  validity  of  such  predictions. 

In  summary,  penalties  and  rewards  will  affect  the  importance  of  successful 
completion  of  the  task.  If  importance  is  greater  than  costs  associated  with 
performing  the  task,  outcome  value  will  be  high,  leading  effort  to  be  expended, 
and  automation  use  to  be  less  likely.  In  some  situations,  this  will  lead  to  an 
optimal  task  allocation  decision;  sometimes,  it  will  lead  to  disuse  of  automated 
aids. 

In  conclusion,  the  framework  of  automation  use  predicts  that  cognitive, 
motivational,  and  social  processes  work  together  to  cause  misuse,  disuse,  and 
appropriate  automation  use.  Many  factors  affect  each  of  the  processes  and  may 
therefore  affect  automation  use.  The  reliability  of  the  automated  aid,  the 
reliability  of  manual  operation,  and  several  cognitive  biases  (including  self- 
serving  and  the  bias  toward  automation)  combine  to  affect  the  perceived  utility 
of  the  aid.  When  the  perceived  utility  of  the  aid  is  high,  the  operator  is  likely  to 
trust  the  aid  and  feel  dispensable;  his  or  her  efforts  are  not  necessary  for  the  task 
to  be  completed.  Automation  use  is  predicted  to  be  high  through  both  social  and 
motivational  processes.  Fatigue,  number  of  tasks,  intrinsic  interest  in  the  task, 
cognitive  overhead,  penalties  for  task  failure,  and  rewards  for  task  completion 
combine  to  affect  the  outcome  value,  which  also  will  affect  the  effort  the  human 
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will  expend  on  the  task  and  the  likelihood  he  or  she  will  rely  on  (he  automated 
aid  (see  Figure  8). 


Future  research  needs  to  determine  the  effect  of  each  of  the  cognitive,  social,  and 
motivational  processes  on  automation  use.  To  examine  one  such  process,  the 
other  two  processes  win  have  to  be  controlled  or  held  constant.  Such  a  procedure 
was  discussed  earlier  in  the  report  (see  pages  4-5)  (Studies  2  and  3  by  Dzindolet 
et  al.,  1999).  In  both  experiments,  researchers  eliminated  the  automation  bias  by 
providing  participants  with  the  automated  aid's  decision  only  after  the  human 
operator  had  indicated  his  or  her  decision.  In  addition,  motivational  processes 
were  controlled.  The  effort  necessary  to  expend  for  a  self-reliance  decision  was 
equivalent  to  the  effort  necessary  to  expend  for  an  automation-reliance  decision. 
Using  this  procedure,  the  researchers  were  able  to  examine  the  social  processes. 
Disuse  was  found  to  be  widespread  and  difficult  to  improve. 

In  addition,  the  interaction  of  the  three  processes  must  be  examined.  Specifically, 
the  relative  importance  of  each  process  needs  to  be  determined.  When  all  three 
processes  were  free  to  impact  automation  use,  Dzindolet  and  others  (to  be 
published)  found  misuse  to  be  more  prevalent  than  disuse.  Would  increasing  the 
penalties  and  rewards  for  correct  decisions  reduce  the  misuse?  How  high  would 
the  consequences  have  to  be  before  the  reduction  in  misuse  became  disuse? 

Controlling  the  automation  bias  and  the  cognitive  processes,  Dzindolet  and 
others  (1999)  found  disuse  rather  than  misuse  to  be  prevalent.  Ostensibly,  the 
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tendency  toward  disuse  found  in  these  studies  was  counteracted  by  the 
automation  bias  in  the  study  by  Dzindolet  and  others  (to  be  published)  to  such 
an  extent  that  misuse  became  more  of  a  problem  than  disuse.  How  much  weight 
does  the  automation  bias  play  in  automation  use?  How  strong  must  the  disuse 
due  to  social  processes  be  to  counteract  the  automation  bias? 

Clearly,  much  research  is  needed  to  examine  this  framework.  We  believe  the 
framework  will  prove  useful  to  researchers  interested  in  reducing  automation 
misuse  and  disuse. 
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